It Doesn't Have to Be Like This: Cybersecurity Vulnerability Trends

It often seems that every newly announced major data breach sets a record for the depth and size of impact. Internet users, nearly everyone these days, naturally wonder: Why is this happening, and how much worse can it get? In the inaugural article for this column, published in January 2009, we reviewed trends in vulnerabilities for the previous eight years [2]. Our goal, then as well as now, is to improve the understanding of cybersecurity vulnerabilities so that we can prevent them. One Moore's Law generation later, we followed that article with another review of trends, finding some encouraging results [3]. In this article, we review some of those earlier findings, plus what has happened since then, and prospects for the near future. Our data source is the US National Vulnerability Database (NVD) [1], which collects nearly all publicly reported vulnerabilities since 1997, using the Common Vulnerabilities and Exposures (CVE) dictionary. It is developed and run by the US National Institute of Standards and Technology, with support from the Department of Homeland Security's National Cyber Security Division. As of 2017, the NVD includes more than 85,000 vulnerabilities, and the collection is expanded daily. With two decades of data, the NVD is an invaluable resource for security analysts. One of the primary observations from the January 2009 analysis was that the total number of vulnerabilities per year had begun to decline, from a peak of nearly 7,000 in 2006 to about 5,500 in 2008. It appeared that developers and security administrators had begun taking security seriously, including it as a key component in development, and staying up to date on mitigation techniques. Code flaws that were widely used in system exploits in the 1980s and 1990s, such as format string vulnerabilities and race conditions, were appearing in only a dozen or two cases each year, accounting for less than 1% each of the vulnerabilities in thousands of applications. Better development methods and tools had begun to make a difference. But the 2009 analysis also revealed a trend that we see repeatedly in all aspects of security new information technology produces new challenges to secure it. During the previous decade, e-commerce and other web-based services had proliferated, producing new challenges for protection and new opportunities for attackers. While buffer overflows and misconfigurations had long been the main sources of weaknesses in systems defenses, SQL injections and cross-site scripting were respectively the #1 and #2 vulnerability types in 2008 (Fig. 1). (Note that the analysis is limited to the distribution of primary vulnerability categories; another 10% 15% each year are classed as either "other" or "insufficient information".) As we will see later in this article, the trends for these two vulnerability types illustrate an important lesson for managing cybersecurity.